An Evaluation of Data Costs for Optical Flow

Motion estimation in realistic outdoor settings is significantly challenged by cast shadows, reflections, glare, saturation, automatic gain control, etc. To allow robust optical flow estimation in these cases, it is important to choose appropriate data cost functions for matching. Recent years have seen a growing trend toward patch-based data costs, as they are already common in stereo. Systematic evaluations of different costs in the context of optical flow have been limited to certain cost types, and carried out on data without challenging appearance. In this paper, we contribute a systematic evaluation of various pixeland patch-based data costs using a state-of-the-art algorithmic testbed and the realistic KITTI dataset as basis. Akin to previous findings in stereo, we find the Census transformation to be particularly suitable for challenging real-world scenes.